Management of Evidence for Compliance with Safety Standards: A Survey on the State of Practice

In many domains such as avionics, railway, and automotive, safety-critical systems must comply with prescribed measures for safety, usually presented in the form of safety standards. Demonstrating compliance with a safety standard involves providing convincing evidence that the requirements envisaged by the standard are adequately met. Real systems are large and thus require collecting and managing large quantities of evidence throughout their lifecycle. Lack of knowledge on how to collect, structure, and assess evidence can lead to certification risks. This paper aims to provide insights into how practitioners deal with safety evidence management. Specifically, we report on a survey conducted with the goal of determining practitioners’ practices and perspectives on safety evidence management. A total of 52 practitioners from 15 countries and 11 application domains responded to the survey, indicating what types of information constitute evidence for safety, how evidence is structured and assessed, how evidence evolution is addressed, and what challenges they face with regards to providing evidence. The paper further analyses the commonalities across different application domains and compares the results against the state of the art. The results notably indicate that practitioners currently undertake an extensive amount of manual work while collecting and manipulating evidence information. This can be both costly and error-prone, thus suggesting the need for better tool-support for evidence management.